SUMMARY OF CONSIDERATIONS TO STRENGTHEN STATE 
REQUESTS FOR ESEA FLEXIBILITY 



During the ESEA flexibility peer review process conducted December 5-9, 2011, peers noted several 
issues that were common across multiple State requests. This document, prepared by U.S. 
Department of Education (Department) staff, provides summaries of peer comments and 
recommendations regarding these cross-cutting issues and suggests additional resources that may be 
helpful to States preparing requests for submission. These summaries and recommendations follow. 



CONSULTATION 

In its request for ESEA flexibility, a State educational agency (SEA) must provide a description of 
how the SEA meaningfully engaged and solicited input on its request from diverse communities and 
appropriate stakeholders. By engaging relevant stakeholders at the outset of the planning and 
implementation process, an SEA can ensure that those stakeholders have input in shaping the SEA's 
request, which will help ensure its successful implementation. 

In the peer review process, peers noted that many SEAs did not provide strong evidence of active 
consultation with teachers and their representatives, or with diverse communities, especially 
organizations representing students with disabilities and English Learners. Peers suggested that an 
SEA actively engage stakeholders in discussions regarding its request rather than merely providing 
notification of the content of the request (such as by posting information regarding the request on a 
Web site). They also noted that multiple SEAs did not provide strong evidence that their flexibility 
requests were presented to parents and other stakeholders in a manner that clearly articulated the 
major changes proposed for their existing accountability systems. Peers raised particular concern 
regarding stakeholder understanding of subgroup accountability changes. 

Peers also noted a few approaches that appeared to help SEAs meaningfully engage and solicit input 
from stakeholders. For example: 

• Some SEAs used multiple methods of communication, including a combination of surveys, 
focus groups, advisory committees, community engagement forums, and outreach to 
teachers within the State. 

• One SEA included multiple stakeholders representing diverse groups on a flexibility working 
group that met several times during the development of the request. 

• Another SEA developed a consultation action plan and assigned specific staff to reach out to 
specific organizations. 

• Other SEAs noted, in their submissions, specific changes that they had made in response to 
stakeholder input. 



PRINCIPLE 1: COLLEGE- AND CAREER-READY EXPECTATIONS FOR ALL 
STUDENTS 



Supporting Content Teachers in Helping Students with Disabilities Achieve College- and 
Career-Ready Standards 

In its request for ESEA flexibility, an SEA must provide a transition plan to implement college- and 
career-ready standards and to ensure teachers are prepared to teach all students to these more 
rigorous standards. Peers noted that the plans for professional development related to teaching 
students with disabilities were generally lacking in detail or comprehensiveness. For example, in 
some SEA plans, professional development related to the needs of students with disabilities would 
be provided only to special education teachers, not to all teachers who teach students with 
disabilities. Additionally, not all SEAs that currently administer alternate assessments based on 
modified academic achievement standards (AA-MAAS) explained how students with disabilities 
taking such assessments would transition to new high-quality assessments, nor how instruction 
would be adjusted to support the transition. 

Peers offered multiple suggestions, some of which build upon what SEAs included in their requests, 
for how an SEA could improve in this area, including: 

• Training all staff to use tiered instructional frameworks and accommodations for learning 
and assessments. 

• Incorporating Universal Design for Learning practices into a State's curriculum design. 

• Specifically addressing the needs of students with disabilities in the State's model curriculum. 

• Developing and providing an integrated professional development plan to ensure that all 
teachers are able to deliver challenging content to students with disabilities using 
instructional strategies that differentiate instruction according to student needs. 

• Planning activities and instructional strategies specifically related to the transition of students 
who now participate in an AA-MAAS to the State's new high-quality assessments. 

• Collaborating with experts and advocates for students with disabilities to identify research- 
based practices to include in the plan for transitioning to college- and career-ready standards. 

Additional Resources: 1 

• National Center on Educational Outcomes (NCEO) (http:/ / www.cehd.umn.edu/nceo) 

• National Comprehensive Center on Teacher Quality ( http:/ /www. tqsource.org) 

• IDEA Partnerships ( http:/ /ideapartnerships.org ) 



1 The links for "additional resources" contained in several places in this document are provided as examples 
of what may be useful resources to States, and are provided for the convenience of the reader. The 
Department does not control or guarantee the accuracy, relevance, timeliness, or completeness of these 
resources, nor does the inclusion of links to these resources represent an endorsement of these resources or 
the organizations that made them available. Readers are invited to review these resources and consider their 
possible use as they determine appropriate. There may be other resources available that might also be useful. 
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Supporting Content Teachers in Helping English Learners Achieve College- and Career-Ready 
Standards 

Implementing college- and career-ready standards means that teaching and learning aligned with 
those standards is taking place in all public schools for all students. To do this work effectively, an 
SEA must support English Learners in reaching rigorous standards. Peers noted that SEAs need a 
clearly articulated plan to build capacity and provide professional development to content teachers 
who teach English Learners and to teachers in language instruction educational programs on how to 
support English Learners in acquiring content knowledge as they develop English language 
proficiency. 

Peers also provided recommendations for strengthening the transition to college- and career-ready 
standards for English Learners, including: 

• Providing professional development for school leaders and all content teachers that includes 
strategies and resources for teaching challenging content when English Learners are not yet 
proficient in the language typically used to explain the content. 

• Creating instructional support materials for reading/language arts, mathematics, and science 
that help teachers address the academic language that English Learners need to access the 
content. 

Additional Resources: 

• Technical Assistance and Dissemination Network (http:/ /www. tadnet.org) 

• Equity Assistance Centers ( http: / / www.equityassistancecenters.org) 

• National Clearinghouse for English Language Acquisition ( http:/ /www.ncela.gwu.edu) 

PRINCIPLE 2: STATE-DEVELOPED SYSTEMS OF DIFFERENTIATED 
RECOGNITION, ACCOUNTABILITY, AND SUPPORT 

Using Multiple Indicators to Measure School Performance 

In their requests for ESEA flexibility, several SEAs proposed the creation of indexes as part of their 
broader accountability systems to measure local educational agency (LEA) and school performance, 
recognize high-performing schools, identify low-performing schools, and target interventions. In 
reviewing the requests, peers expressed concern that some of the indexes were complex and 
included many factors and calculations that make it difficult for parents and educators to understand 
the reason for the rating or grade designation of a school. In addition, peers thought that the 
inclusion of multiple, compensating measures could make it more difficult for SEAs to use the 
indexes to differentiate among schools. For example, some peers had concerns about the weights 
given to the factors in indexes, particularly where graduation rates appeared to play a fairly minor 
role or where the index did not sufficiently emphasize achievement measures. 

In light of these concerns, an SEA should consider what it is trying to accomplish with its index and 
then determine the key measures and the weights to be given those measures within the index. An 
SEA should also consider its goals for improving school performance and the incentives it wants to 
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create, and then determine whether it can accomplish these goals and provide these incentives 
through a simpler structure. 

For an SEA that chooses to incorporate a complex index into its accountability system, peers 
provided technical assistance suggestions, including: 

• Providing clear explanations of how the many factors in the index roll up into an 
accountability determination. 

• Ensuring that the index provides sufficient differentiation of schools. 

• Giving significant emphasis to measures of student achievement, growth, and graduation 
rates. 

• Developing materials and systems to explain connections among index scores, observable 
outcomes, and strategies schools could use to improve teaching and learning. 

• Giving adequate weight in the index to subgroup performance, not just overall performance, 
to ensure subgroup accountability. 

A separate but related concern peers noted is that multi-measure indexes that give lesser weight to 
graduation rates may unintentionally weaken graduation accountability. For example, this would be 
the case if a school could make progress on an index for a number of years without improving its 
graduation rate. 

Peers suggested a number of options to strengthen graduation rate accountability, including: 

• Including graduation rates disaggregated by subgroup in the index. 

• Increasing the weight of the graduation rate measure in the index to be more closely 
balanced with achievement scores. 

• Weighting school completion in an index by including a measure of dropout rate or 
including a zero score for each dropout in the achievement measure. 

• Using the graduation rate as a separate accountability measure in conjunction with the index 
scores. 

Including the Performance of Consolidated Subgroups in Accountability Systems 

Several SEAs proposed the creation of a "consolidated subgroup" that would combine multiple 
ESEA subgroups — i.e., major racial and ethnic groups, economically disadvantaged students, 
students with disabilities, and English Learners — for accountability purposes, in addition to 
proposing the continued use of the individual ESEA subgroups. Under ESEA flexibility, an SEA 
must continue to set annual measurable objectives (AMOs) and report on progress against those 
targets for all ESEA subgroups. In addition, an SEA must use those AMOS, as well as other 
measures, to provide incentives and supports to ensure continuous improvement in Title I schools 
not identified as priority or focus schools that are not making progress in improving student 
achievement and narrowing achievement gaps. However, an SEA may propose combining 
subgroups as part of its broader accountability system or in order to identify focus schools, as long 
as it continues to include the performance of all ESEA subgroups in its accountability system and it 
demonstrates that its use of a consolidated subgroup is educationally sound. Such a demonstration 
might include the presentation of data showing that the SEA's use of a consolidated subgroup holds 
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more schools accountable for the performance of students belonging to ESEA subgroups than 
would otherwise be held accountable for their performance by combining small subgroups that fall 
under the SEA's minimum n-size. 

Peers were concerned with some of the ways that SEAs created and used consolidated subgroups. 
For example, they noted that a consolidated subgroup, particularly one that consolidated 
racial/ ethnic groups or "high-needs" subgroups, could allow a higher-performing group within the 
consolidated subgroup to mask the poor performance of other groups, or could mask the 
performance of one consistently low-performing group. Peers were also concerned that some uses 
of a consolidated subgroup would not ensure accountability for the performance of the subgroups 
identified in the ESEA. Therefore, peers advised that an SEA that employs a consolidated subgroup 
include mechanisms, or safeguards, to ensure that the performance of each individual ESEA 
subgroup is not masked by the consolidated subgroup, and that the needs of the students in each 
subgroup are addressed. There are a number of ways that an SEA could do this, including: 

• Identifying as focus schools those schools in which a single subgroup is performing 
substantially below the State average, and requiring those schools to improve the 
performance of that subgroup before exiting focus status. 

• Using AMOs for all ESEA subgroups as an element in an index or as a factor in a school's 
grade or to identify schools that are not priority and focus schools. 

• Identifying LEAs and schools for the most intensive interventions when a single ESEA 
subgroup does not make progress on a majority of measures. 

• Requiring interventions in any school in which an ESEA subgroup misses AMOs over a 
number of years or multiple ESEA subgroups miss AMOs, and using the number of AMOs 
missed as a factor in assigning school grades. 

In general, peers suggested that an SEA establish safeguards to ensure appropriate attention is given 
and action is taken when one or more subgroups are not meeting goals even if the school is making 
progress on its index measure or for a consolidated subgroup. 

Using AMOs for Continuous Improvement in Other Title I Schools 

One of the requirements of ESEA flexibility is that AMOs be used, along with other measures, to 
inform incentives and supports in Title I schools not identified as priority or focus schools. In 
reviewing SEA requests, peers noted significant weaknesses in this area. 

Peers also identified examples of strong approaches to using AMOs to inform incentives and 
supports that an SEA might consider: 

• Using individual subgroup performance on AMOs as an element of an SEA's index. 

• Identifying and requiring LEA-determined interventions in schools that are not identified as 
priority or focus schools but in which an individual subgroup does not meet its AMOs for 
multiple years. 

• Using subgroup performance on AMOs to differentiate school performance within a school 
grade category defined by an index system. For example, an SEA might use pluses and 
minuses on the State's grading system (e.g., a school that is otherwise an "A" school but 
misses a certain number of AMOs is an ' A-"; a school that is otherwise a "C" school but 
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misses only a certain number of AMOs is a "C+"), and differentiate incentives and support 
based on the pluses and minuses. Only schools that meet all AMOs receive an "A+". 

• Using whether subgroups make AMOs to focus the delivery of support services to schools 
in order to close achievement gaps for all subgroups. 

Including Students with Disabilities in Accountability Systems and Providing Interventions to 
Meet Their Needs 

Under ESEA flexibility, the State-developed system of differentiated accountability must create 
incentives and provide supports to close achievement gaps for all subgroups of students and include 
interventions focused on improving the performance of students with disabilities. Peers noted that, 
in some of the requests they reviewed, the supports and interventions cited for priority, focus, and 
other Title I schools did not focus on students with disabilities; instead, interventions were often 
generic rather than specific to the needs of subgroups. Additionally, some of the strategies included 
for students with disabilities emphasized the use of remedial and developmental programs, rather 
than rigorous instruction for all students, and, in some requests, there was a lack of focus on 
improving content instruction for all students in all classrooms. Peers also indicated that students 
with the most significant cognitive disabilities who participate in a State's alternate assessment based 
on alternate academic achievement standards (AA-AAAS), as well as other students with disabilities, 
were not always included in the accountability system in the same manner as all other students. 

Peers offered multiple suggestions, some of which build upon what SEAs included in their requests, 
for how an SEA could improve in this area, including: 

• Ensuring that all students with disabilities are included in the performance data used to 
identify schools and implement interventions. 

• Including students with disabilities in both the proficiency and gap-reduction components of 
an accountability system index. 

• Including descriptions of research-based interventions specifically designed for students with 
disabilities in an SEA's request. 

• Identifying options to include the progress of students with the most significant cognitive 
disabilities who participate in an AA-AAAS in accountability systems. 

• Creating a single resource that describes available interventions, including research-based 
interventions for students with disabilities, and how the SEA expects LEAs to apply those 
interventions in priority, focus, and other Title I schools. 

• Explaining in the SEA's request the relevance of specific interventions and resources for 
improving achievement for all students and subgroups and how progress will be tracked and 
evaluated. 

• Implementing an early warning system so that struggling students are identified early and 
provided interventions through a response to intervention-like structure. 

• Strengthening the school improvement planning process and conducting a needs analysis to 
identify the reasons for persistent low performance among subgroups. 

• Identifying successful practices that accelerate student learning as opposed to offering only 
remedial and developmental instruction. 
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Additional Resources: 

• Telfer, D.M. (201 1). Movingjour numbers: Five districts share how thej used assessment and 
accountability to increase performances for students with disabilities as part of district-wide improvement. 
Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. 

• National Center on Educational Outcomes ( http:/ / www.cehd.umn.edu/nceo) 

• National Center on Response to Intervention ( http:/ /www.rti4success.org) 

• The IRIS Center for Training Enhancements ( http: / / iris.peabody.vanderbilt.edu ) 

• Universal Design for Learning Center ( http:/ /www. udlcenter.org ) 

• Institute of Education Sciences: What Works Clearinghouse 
(http:/ /ies.ed.gov/ncee/wwc/publications reviews.aspx) 

• The National Center on Scaling Up Effective Schools 
(http: / / www.scalingupcenter.org/ research/index.aspx) 

• IDEA Partnerships ( http:/ /ideapartnerships.org) 

Including English Learners in Accountability Systems and Providing Interventions to Meet 
Their Needs 

As described above, an SEA's accountability system must create incentives and include differentiated 
interventions and supports to improve student achievement and graduation rates and to close 
achievement gaps for all subgroups, including interventions specifically focused on improving the 
performance of English Learners. Peers expressed concerns that not all SEAs had provided specific 
interventions that adequately address the needs of English Learners or explained how services to 
these students would help to reduce achievement gaps. To address this issue, peers recommended 
that an SEA develop a well-defined plan on how schools and teachers will help English Learners 
transition to rigorous standards that includes identification of the linguistic demands of the SEA's 
new content standards, professional development for teachers, and formative assessments that 
measure the progress of English Learners. 

Peers noted several ways that an SEA plan could address this area, including: 

• Developing a coherent plan to provide professional development for all teachers on 
addressing English Learners' needs. 

• Developing intervention plans that provide for the delivery of research-based interventions 
tied to the specific needs of English Learners. 

• Creating a single resource that describes available interventions, including research-based 
interventions for English Learners, and how the SEA expects LEAs to apply those 
interventions in priority, focus, and other Title I schools. 

• Developing a stronger system of accountability for low subgroup performance in all schools, 
and particularly for Title I schools not identified as priority or focus schools. 

• Explaining in the SEA's request the relevance of specific interventions and resources for 
improving achievement for English Learners and how their progress will be tracked and 
evaluated. 
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• Strengthening the school improvement planning process and conducting a needs analysis to 
identify the reasons for persistent low performance among English Learners. 

• Identifying successful practices that accelerate content learning as opposed to offering only 
remedial and/ or English language development instruction in isolation. 

PRINCIPLE 3: SUPPORTING EFFECTIVE INSTRUCTION AND LEADERSHIP 

Strengthening Teacher and Principal Evaluation and Support Systems 

Peers had several general concerns regarding SEAs' proposals to develop and implement teacher 
and principal evaluation and support systems. These include, for example, lack of specificity 
regarding the process for determining the validity and reliability of the evaluation measures and how 
they will be consistently applied across LEAs. Further, peers had concerns related to the measures 
SEAs intended to use to evaluate teachers of non-tested grades and subjects. 

Numerous resources are available to address these concerns, including the following: 

• A Practical Guide to Designing Comprehensive Teacher Evaluation Systems (National 
Comprehensive Center for Teacher Quality, available at: 

http:/ /www. tqsource.org/publications/practicalGuideE valSystems.pdf) . 

• Measuring Student Growth for Teachers in Non-Tested Grades and Subjects: A Primer 
(Reform Support Network, available at: 

http://www.swcompcenter.org/educator effectiveness2/NTS PRIMER FINAL.pdf) . 

• Great Teachers and Leaders: State Considerations on Building Systems of Educator 
Effectiveness (Reform Support Network, available at: 

http:/ / www2.ed.gov/ programs /racetothetop/ great-teachers.doc) . 

• Alternative Measures of Teacher Performance (National Comprehensive Center for Teacher 
Quality, available at: http:/ /www.tqsource.org/pdfs/TQ Policy-to- 
PracticeBriefAlternativeMeasures.pdf ). 

• Guide to Teacher Evaluation Products (National Comprehensive Center for Teacher 
Quality, available at: http:/ /www3 .learningpt.org/ tqsource/ GEP) . 

• Measuring Teachers Contributions to Student Learning Growth for Non-tested Grades and 
Subjects (National Comprehensive Center for Teacher Quality, available at: 

http:/ /www.tqsource.org/publications/MeasuringTeachersContributions.pdf ). 

• State Policies and Examples of Best Practices in Principal Evaluation (National 
Comprehensive Center for Teacher Quality, available at: 

http:/ /www. tqsource.org/webcasts/201106Workshop/Presentations/ ConcurrentSessionl 
StatePoliciesInPrincipalEval.pdf ). 

• Getting It Right: A Comprehensive Guide to Developing and Sustaining Teacher 
Evaluation and Support Systems (National Board for Professional Teaching Standards, 
available at: http:/ /www.nbpts.org/userfiles/file/NBPTS Getting-It-Right.pdf ). 

• Labor-Management Collaboration Conference Toolkit (U.S. Department of Education, 
available at: http:/ /www.ed.gov/ sites/ default/ files /lmc-conference-toolkit.pdf ). 
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Although some of these resources are designed specifically for Race to the Top grantees, the 
Department believes the information they contain may be useful to SEAs and LEAs that implement 
ESEA flexibility. 

Evaluating Teachers of English Learners and Students with Disabilities 

Peer reviewers identified multiple weaknesses in SEAs' plans to evaluate teachers of English 
Learners and students with disabilities. For example, peers observed that in some SEAs' requests 
there was not a systematic process to solicit input on the guidelines for evaluating these teachers. 
Additionally, SEAs did not provide strong evidence that the rubrics for training and evaluating 
teachers and principals addressed the education of English Learners and students with disabilities. 
Peers expressed concerns that SEAs' plans did not specify how the performance rating system will 
include teachers of English Learners or students with disabilities in a meaningful way when they may 
teach those students part of the time, teach multiple classes, or serve as resource teachers. 

Peers offered multiple suggestions, some of which build upon what SEAs included in their requests, 
for how an SEA could improve in this area, including: 

• Including teachers in language instruction educational programs and special education 
teachers, as well as experts in measuring growth for students with disabilities and English 
Learners, in evaluation workgroups. 

• Developing guidance regarding evaluations based on student learning objectives for students 
who participate in an AA-AAAS. 

• Developing evaluation guidelines for teachers who do not have primary responsibility for 
content instruction for English Learners and students with disabilities (e.g., teachers in 
language instruction educational programs and special education teachers). 

• Ensuring teachers and principals have input and are engaged in a meaningful way in the 
piloting of an evaluation system. Teachers who teach students with disabilities and English 
Learners, in particular, should be included in this effort. 

• Creating data linkages and teacher-of-record policies that lead to inclusion of students with 
disabilities and English Learners in teacher evaluations. 

• Developing rubrics and evaluator training activities that include pedagogical best practices in 
teaching students with disabilities and English Learners. 

• Making the needs of students with disabilities and English Learners explicit in educator 
practice rubrics and in training evaluators to assess teachers' skills. 

• Determining how teachers who provide part-time instructional services to students with 
disabilities and English Learners will be evaluated. 

• Using growth measures that include students with disabilities and English Learners. 

Additional Resources: 

• National Center on Educational Outcomes (NCEO) ( http:/ / www.cehd.umn.edu/nceo) 

• National Clearinghouse for English Language Acquisition ( http:/ / www.ncela.gwu.edu) 

• Technical Assistance and Dissemination Network (http:/ /www. tadnet.org) 

• Equity Assistance Centers ( http: / / www.equityassistancecenters.org) 

• IDEA Partnerships ( http:/ /ideapartnerships.org) 
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